Automatinio mokinių programų vertinimo sistemų lyginamoji analizė

Mokant programavimo jau seniai taikomas automatinis vertinimas. Tokiu atveju mokiniai (ar studentai) turi programas-sprendimus pateikti testavimo sistemai. Kad sistema galėtų patikrinti darbus, jie turi tenkinti gana griežtus reikalavimus. Savo ruožtu automatinė vertinimo sistema gali įvertinti dalį pamatuojamų kriterijų, patikrinti programos funkcionavimą su pradinių duomenų rinkiniais. Automatinių sistemų funkcionalumas kartais praplečiamas rankinėmis vertinimo galimybėmis, taip gaunantpusiau automatines sistemas. Sistemose paplitę du pagrindiniai programų analizės metodai: dinaminis ir statinis, o vertinimo būdų, kriterijų įvairovė yra gerokai didesnė. Automatinės mokinių programų vertinimo sistemos gelbsti ne tik vertinant kontrolines užduotis. Jos gali turėti ir mokomąją funkciją – mokinys gauna galimybę greitai, vos ne bet kuriuo momentu pasitikrinti, gauti kitą panašią užduotį, esti įsitikinęs vertinimo objektyvumu. Siekiant tobulinti automatinio vertinimo mokomąją funkciją reikia skirti daugiau dėmesio esamų sistemų analizei, jų pranašumams ir trūkumams. Straipsnyje lyginamosios analizės būdu vertinami sprendimai, siūlomos naujos sistemų tobulinimo sritys.Comparative Analysis of Automatic Students’ Program Evaluation SystemsBronius Skūpas SummaryPractice in programming is typical for graduators of secondary schools and for programming courses in universities. Students usually have programming assignments that need to be assessed. The assessment can be done using automatic assessment systems. There are several areas where such systems can be used: programming competitions and preparations for them, evaluation of maturity exam in programming, teaching of programming in courses. Static and dynamic assessment of programs is discussed. Article compares several different assessment systems described in literature and available from Internet. Comparative analysis shows main strength and weakness of automatic assessment systems. Requirements, possibilities and trends for future assessment systems are discussed

Pasikeitimų informacinių technologijų valstybiniame brandos egzamine analizė

2011 metais informacinių technologijų valstybinio brandos egzamino vykdymo tvarka buvo esmingai pakeista – įvesta galimybė praktinę užduotį atlikti ne tik Paskalio, bet ir C++ programavimo kalba. Šis pakeitimas ne tik suteikia naujų galimybių mokiniams, bet ir sukelia tam tikrų vertinimo proceso problemų. Straipsnyje analizuojami būsimi vertinimo proceso pokyčiai, projektuojami reikiami pusiau automatinės vertinimo sistemos adaptavimo naujai situacijai poreikiai, prognozuojamos galimos vertinimo problemos ir teikiami objektyvesnio vertinimo pasiūlymai. Pirmasis egzaminas pagal naujas vykdymo ir vertinimo instrukcijas įvyko 2011 m. birželio 3 d. Analizuojama, kokią įtaką prognozuotos problemos turėjo egzamino vertinimui.Pagrindiniai žodžiai: informacinės technologijos, valstybinis brandos egzaminas, pusiau automatinis vertinimas.Analysis of Changes in the National Maturity IT ExamBronius Skūpas SummaryThe national maturity IT exam was changed in 2011 by adding the possibility to use C and C++ programming languages in practical assignments. Before 2011, it was possible to use only the Pascal programming language. This change leads to wide possibilities for students and involves great changes in the evaluation process as well. The semi-automatic evaluation system must be changed accordingly, and more attention to the selection of tasks, tests and even evaluators must be given. The author analyses the necessary changes in the evaluation process and system. He predicts the possible problems, analyses them and provides their possible solutions. The first exam of this style took place on 3 June 2011. Its results are compared with the prognosis

Automatinio ir pusiau automatinio vertinimo ypatumai IT valstybiniame brandos egzamine

National IT maturity exam in Lithuania have important practical programming part. Task solutions are evaluated in automatic and semi-automatic way. Article analyses reasons for semi-automatic evaluation in exam. Good programs with poor automatic testing results are analysed also and errors are classified.Informacinių technologijų valstybinis brandos egzaminas (ITVBE) pasižymi specifine savybe – praktinėmis programavimo užduotimis, atliekamomis prie kompiuterio. Užduočių sprendimai vertinami tiek automatiniu, tiek pusiau automatiniu būdu. Straipsnyje analizuojamos priežastys, kurios verčia taikyti pusiau automatinį vertinimą šio egzamino vertinime. Tiriamos gerų programų, neįveikusių automatinio testavimo testų, būdingos klaidos

Automatinių vertinimo sistemų grįžtamojo ryšio tobulinimas

Automatic program evaluation is a way to assess source program files. These techniques are used in learning management environments, programming exams and contest systems. However, use of automated program evaluation encounters problems: some evaluations are not clear for the students and the system messages do not show reasons for lost points. The author proposes several ideas for possible improvements in black box testing, which can lead into better service for the users of automatic evaluation systems

Pusiau automatinio programavimo užduočių vertinimo ir testavimo metodas

The thesis investigates the automatic and semi-automatic evaluation of programming assignments in teaching of programming, exams and competitions. Manual evaluation of programming assignments is still dominant in programming teaching in Lithuania. In some contexts (i.e. programming competitions, basics of programming and algorithms course) there is widely applied automatic and semi-automatic evaluation of programming tasks based on black-box testing approach. However, automatic evaluation based on black-box testing is criticized for its inability to demonstrate the strengths and the weaknesses of the program being evaluated. Automatic evaluation is not capable of identifying the level of achievement in incomplete programs. The thesis addresses the problem of evaluation of practical programming assignments in Information Technology maturity exam (IT VBE). This problem involves requirements to evaluate reliably thousands of programming assignment solutions in a reasonable amount of time. The thesis presents analysis of systems for automatic evaluation of programming tasks and appropriate types of assignments. The analysis led to the improvement of the evaluation method. The most important result of the research is the proposed way to improve the semi-automatic evaluation and testing method by increasing its interactivity. A semi-automatic assessment system is expanded by adding the components responsible for the modification of submitted programs, monitoring changes in the functionality of the modified program. The proposed method has been tested in practice - it was used in the IT VBE practical task assessment system developed by the author. This tool was used to test the effectiveness and the quality of evaluation by applying the proposed method. The analysis of possibilities of applying the proposed method to other semi-automatic evaluation systems was conducted

A Method for Semi-Automatic Evaluation and Testing of Programming Assignments

The thesis investigates the automatic and semi-automatic evaluation of programming assignments in teaching of programming, exams and competitions. Manual evaluation of programming assignments is still dominant in programming teaching in Lithuania. In some contexts (i.e. programming competitions, basics of programming and algorithms course) there is widely applied automatic and semi-automatic evaluation of programming tasks based on black-box testing approach. However, automatic evaluation based on black-box testing is criticized for its inability to demonstrate the strengths and the weaknesses of the program being evaluated. Automatic evaluation is not capable of identifying the level of achievement in incomplete programs. The thesis addresses the problem of evaluation of practical programming assignments in Information Technology maturity exam (IT VBE). This problem involves requirements to evaluate reliably thousands of programming assignment solutions in a reasonable amount of time. The thesis presents analysis of systems for automatic evaluation of programming tasks and appropriate types of assignments. The analysis led to the improvement of the evaluation method. The most important result of the research is the proposed way to improve the semi-automatic evaluation and testing method by increasing its interactivity. A semi-automatic assessment system is expanded by adding the components responsible for the modification of submitted programs, monitoring changes in the functionality of the modified program. The proposed method has been tested in practice - it was used in the IT VBE practical task assessment system developed by the author. This tool was used to test the effectiveness and the quality of evaluation by applying the proposed method. The analysis of possibilities of applying the proposed method to other semi-automatic evaluation systems was conducted

Get involved! The IOI workshop 2010, its goals and results

In May 2010, the third IOI workshop took place in Schloss Dagstuhl, Germany. It was motivated by the discussions held at and after the panel session of 2009's IOI conference in Plovdiv. There, discussions focussed on communication and collaboration among the IOI community, as well as communication of the IOI competition to outsiders. At the workshop, members of the IOI community met to develop a first version of an IOI Wiki as a tool for communication and collaboration, and devised suggestions on how to visualize IOI-style contests to make them more accessible to the outside world